(12) INTERNATIONAL APPLICATION PUBLISHED UNDER THE PATENT COOPERATION TREATY (PCT) 



(19) World Intellectual Property Organization 
International Bureau 

(43) International Publication Date 
14 March 2002 (14.03.2002) 




PCT 



(10) International Publication Number 

WO 02/21841 Al 



(SI) International Patent Classification^: H04N7/I8, 

G06F3/00, 15/16, H04J 1/02 

(21) International Application Number: PCT/USOl/22002 

(22) International Filing Date: 1 1 July 2001 (1 1.07.2001) 

(25) Filing Language: Ttnglish 

(26) Publication Language: Engli.sh 

(30) Priority Data: 

09/653,964 1 September 2000 (01 .09.2000) US 

(71) Applicant: REARDEN STEEL, BSC. [US/US]: 151 Uni- 
versity Avenue, Palo Alto, CA 94301 (US). 

(72) Inventor: PERLMAN, Stephen, G.; 1 107 Hamilton Av- 
enue, Palo Alto, CA 94301 (US). 

(74) Agents: MALLIE, Michael, J. et al.; Blakely, Sokoloff, 
Taylor & Zafman LLP, 7th Hoor, 12400 Wilshire Boule- 
vard, Los Angeles, CA 90025 (US). 



(81) Designated States (national): AE, AG, AL, AM, AT, AU, 

AZ, BA, BB, BG, BR, BY, BZ, CA, CH, CN, CR, CU, CZ, 
DE, DK, DM, DZ, EE, ES, FI, GB, GD, GE, GH, GM, HR, 
HU, ID, IL, IN, IS. IP, KE, KG, KR KR, KZ, LC, LK, LR, 
LS, LT, LU, LV, MA, MD, MG, MK, MN, MW, MX, MZ, 
NO, NZ, PI., PT, RO, RU, SD, .SE, SG, SI, SK, SL, TJ, TM, 
TR, Ti, rz, UA, UG, UZ, VN, YU, ZA, ZW. 

(84) Designated States (regional): ARIPO patent (GH, GM, 
KB, LS, MW, MZ, SD, SL, SZ, TZ, UG, ZW), Eurasian 
patent (AM, AZ, BY, KG, KZ, MD, RU, TJ , TM), European 
patent (AT, BE, CII, CY, DE, DK, ES, FI, FR, GB, GR, EE, 

IX LU, MC, NI., PT, SE. TR), OAPI patent (BR BJ, CP, 
CG, CI, CM, GA, GN, GW, ML, MR, NR, SN, TD, TG). 

Published: 

wilh iniernalional search report 
— before the expiration of the time limit for amending the 
claims and to be republished in the event of receipt of 
amendments 

For t^vo-letler codes and other abbreviations, refer to the "Guid- 
ance Notes on Codes and A bbreviations " appearing at the begin- 
ning of each regular issue of the PCT Gazette. 



(54) Title: MULTIMEDIA AND COMPUTING SYSTEM 



/pSTN ^ 




Broadcast 
130 



Internet 
120 



00 
tH 

o 
O 



f 



1S1 



NOC 
180 




160 



163 



(57) Abstract: A home media server (1 10) is disclosed comprising: means for receiving broadcast multimedia content (130); means 
for receiving packet-.<!witched multimedia content (120); means for converting the broadcasting multimedia content and said packet 
switched multimedia content into a standard multimedia content format; and means for distributing the multimedia content in the 
standard format to a plurality of multimedia nodes (164, 165, 166, 151, 160, 161, 162, 163, 150). 



wo 02/21841 



PCT/USOl/22002 



Multimedia and Computing System 
BACKGROUND 

Field of the Invention 

This invention relates generally to the field of multimedia systems. More 
particularly, the invention relates to a multimedia system for storing and coordinating the 
distribution of multimedia content and data over local and remote communication channels. 

Description of tibie Related Art 

Thirty years ago, only a limited number of multimedia devices were available to 
consumers. These included, for example, radio receivers for playing AM/FM radio, 
television sets for displaying broadcast video programs, turntables for playing vinyl records, 
and standard telephones for communicating across long distances. These devices were 
relatively easy to use and were the only devices capable of playing the media for which they 
were designed (e.g., a standard LP record could only be played back on a turntable). 

Since that time, the digital revolution has produced a seemingly unlimited number of 
new multimedia devices. For example, audio and video content today may be digitally 
encoded on compact disks ("CDs"), digital video disks ("DVDs"), MiniDisks™, digital 
audio/video tapes, and VHS tapes. Moreover, with the proliferation of high speed Internet 
access through digital subscriber lines ("DSL") and digital cable systems, consumers are 
frequently turning to the Internet to retrieve, store and play back audio and video content 
(e.g., in "MPS" or "MPEG-2" format, respectively). 
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In addition, the number of multimedia channels over which consumers receive audio, 
video, and data has increased significantly. Today these include standard broadcast 
television; digital/analog cable television; various direct-to-home satellite broadcast systems 
(e.g., DirecTV™, the Dish Network™); Internet service via dial-up, DSL and cable; AM/FM 
radio; and standard telephone service. 

As a result, consumers are burdened with coordinating a variety of incompatible 
multimedia types and multimedia communication channels. This is not merely burdensome, 
but also costly and inefBcient in that consumers are required to purchase a variety of 
different stand-alone decoder/playback and encoder/recorder devices, and subscribe to a 
plurality of incompatible multimedia services (e.g., standard telephone service, digital cable 
service, DSL Internet service, . . . etc). 

In addition to the foregoing problems experienced by consumers, serious problems 
have arisen in recent years for artists and other content creators. More specifically, 
copyright infringement has become rampant due to the fact that digital audio and video can 
easily be copied from one media type to another without any noticeable loss in signal quality 
(e.g., from the Internet to a computer hard drive to a recordable compact disk). 

Current device integration and home networking solutions such as those designed by 

the Universal Plug and Play Forum ('TJPNP"), and Jini™ provide some level of 

interconnectivity between digital devices in the home. However, these solutions focus on 

connecting normally autonomous, stand-alone digital devices on a peer-to-peer network and, 

as such, often result in no cost savings to the end user. For example, the devices themselves 
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are relatively expensive because they must be "smart" enough to function outside of the 
network environment and known network resources. In addition, these solutions can be 
extremely difficult to install and configure due to the fact that each device must be equipped 
with a proprietary interface for communicating to other devices on the network. 

Accordingly, what is needed is a centralized, comprehensive, open system capable of 
storing and coordinating between a variety of different media types and media transmission 
channels. What is also needed is a system and method which will protect copyright holders' 
interests in the underlying audio/video material. What is also needed is a system and 
method which will work seamlessly with both currently available standalone devices (e.g., 
personal computers, televisions) as well as non-standalone devices (e.g., Web terminals and 
network telephones). 

SUMMARY OF THE INVENTION 

A home media server is disclosed comprising: means for receiving broadcast 
multimedia content; means for receiving packet-switched multimedia content; means for 
converting the broadcast multimedia content and said packet switched multimedia content 
into a standard multimedia content format; and means for distributing the multimedia 
content in the standard format to a plurality of multimedia nodes. 

BRIEF DESCRIPTION OF THE DRAWINGS 

A better understanding of the present invention can be obtained from the following 

detailed description in conjunction with the following drawings, in which: 
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FIG. 1 illustrates a network architecture for implementing embodiments of the 
invention. 

FIG. 2a illustrates one embodiment of a home media server hardware architecture. 

FIG. 2b illustrates one embodiment of a home media server software architecture. 

FIG. 3 illustrates a plurality of media modules installed in one embodiment of a 
home media server. 

FIG. 4 illustrates one embodiment of a home media server which includes a 
DVD/CD/CD-RW drive. 

FIG. 5 illustrates a home media server communicating to two audio/video media 
nodes over a wireless network. 

FIG. 6a illustrates one embodiment of a multimedia node hardware architecture. 

FIG. 6b illustrates one embodiment of a multimedia node software architecture. 

FIG. 6c illustrates one embodiment of the system for tuning between multimedia 

channels. 

FIG. 6d illustrates one embodiment of the system for playing a CD jukebox. 

FIG. 7a illustrates one embodiment of a home media server for coordinating 
between standard telephone services and IP telephone services. 
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FIG. 7b illustrates one embodiment of a software architecture for implementing 
standard telephone, IP telephone and/or video conferencing. 

FIG. 7c illustrates a signal diagram of communication between a home media server 
and telephone. 

FIG. 8a illustrates a home media server coordinating between standard broadcast 
channels and packet-switched channels (e.g., the Internet). 

FIG. 8b illustrates a potential progression of bandwidth allocation between packet 
switched chaimels and analog/digital broadcast channels. 

FIG. 8c illustrates multimedia buffering according to one embodiment of the 
invention. 

FIG. 9a illustrates a histogram showing a normalized bitrate for a particular 
multimedia stream. 

FIG. 9b illustrates one embodiment of a system for intelligent bandwidth allocation 
and buffering. 

FIG. 9c illustrates bitrate data normalized at one second intervals. 

FIG. 9d illustrates bitrate data for three separate multimedia streams normalized at 
10 second intervals. 

FIGS. 9e-m illustrate histograms of normalized bitrate data for various DVDs. 
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FIG. 10 illustrates a cable television module according to one embodiment of the 
invention. 

FIG. 11 illustrates a satellite module according to one embodiment of the invention. 

FIG. 12 illustrates a cable modem module according to one embodiment of the 
invention. 

FIG. 13 illustrates copyright tags implemented in one embodiment of the invention. 

DETAILED DESCRIPTION 

In the following description, for the purposes of explanation, numerous specific 
details are set forth in order to provide a thorough understanding of the present invention. It 
will be ^parent, however, to one skilled in the art that the invention may be practiced 
without some of these specific details. Li other instances, well-known structures and devices 
are shown in block diagram form to avoid obscuring the underlying principles of the 
invention. 

Embodiments of a Centralized Multbiedia 
AND Computing System 

System Overview 

As illustrated in Figure 1, in one embodiment of the invention, a digital media server 
1 10 equipped with a processor and a mass storage device acts as a central repository for 
decoding, storing and distributing multimedia content and data. More particularly, the 
digital media server 100 coordinates multimedia content from Internet communication 
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channels 120 (e.g., DSL, cable Internet), broadcast communication channels 130 (e.g., 
digital/analog cable, satellite), and/or Public Switched Telephone Network ("PSTN") 
communication channels 170 (i.e., standard telephone) to provide a stable, real-time home 
media network 140 for a plurality of network devices 150-151, 160-166. 

One Embodiment of a Home Media Server Architecture 
As illustrated in Figure 2a, one embodiment of a home media server 1 10 computing 
architecture includes a central processing unit 200 capable of processing data and 
multimedia content stored in main memory 201 and a mass storage device 230 for storing 
data and multimedia content In one embodiment, the central processing unit 200 is a 
Pentium®-class processor such as a Pentium HI® operating at a 1 GHz or faster clock 
frequency. It should be noted, however, that the underlying principles of the invention are 
not Limited to any particular processor speed or processor type. The main memory 201 may 
be a random access memory or any otha- dynamic storage medium (e.g., SDRAM, 
DDRAM, RD-RAM, . . . etc). The mass storage device 230 of one embodiment is capable 
of storing hundreds, or even thousands of hours of multimedia content (e.g., movies, digital 
audio, . . . etc) as well as other types of digital data (e.g., comtputer programs, word 
processing documents, . . . etc). Devices transmit and receive data to/from the mass storage 
device 230 over a high speed interface such as an enhanced IDE interface wifli Ultra DMA 
capabilities or a Small Computer System Interface ("SCSI"). However, various other 
interfaces may be employed while still complying with the underlying principles of the 
invention. 
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An application-specific integrated circuit ("ASIC") 210 coordinates conununication 
between the various system components and ofQoads certain designated processing tasks 
jfrom the CPU. The ASIC may be custom built based on the requirements of the home 
media server 1 10 or may be buUt using gate arrays, standard cells or programmable logic 
devices. 

Communication modules 240-245 electrically coupled to the home media server 1 10 
via a system bus 220, allow the home media server 1 10 to communicate over different local 
and remote communication channels. In one embodiment, the system bus 220 is a 
peripheral component interconnect ("PCI") bus, although various other bus types may be 
configured within the home media server 110 (e.g., ISA, EISA, Micro Channel, VL-bus . . . 
etc). 

In the particular embodiment illustrated in Figure 2a, the commxmication modules 
240-245 electrically coupled to the system bus 220 include an RF network module 240 for 
communicating over the home media network 140 (i.e., via a wireless RF chaimel), a cable 
TV module 241 for receiving broadcast cable channels, a cable modem module 242 for 
providing Internet access via a cable system (i.e., using the TCP/IP protocol), a satellite TV 
module 243 for receiving satellite broadcasts, and a DSL module 244 for DSL Internet 
access. Moreover, a virtually unlimited number of new modules may be added as necessary 
to support new or existing communication channels/protocols (as indicated by module 245). 

Other components within the home media server 1 10 architecture include an MPEG- 

2 decode module 202 (and/or other decode modules such as ACS, MPEG-1, . . . etc); an 
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audio module 203 comprised of a digital-to-analog converter, a Sony-Philips Digital 
Mterconnect Format ("SP-DIF') interface and a standard telephony interface for providing 
digital and analog audio and standard telephone service to external audio/telephony devices; 
an Ethernet port provided directly the system ASIC 210 (as indicated by the "100 Base-T 
Ethernet" designation); a Firewire (IEEE 1394) port 204; a Universal Serial Bus ("USB") 
port 205; and an infrared port 206. Various other communication interfaces may be 
configured in the system, either directly on the primary home media server architecture 1 10 
(e.g., on the media server 110 "motherboard"), or as an add-on module 240-245. Moreover, 
the communication modules (e.g., 202-206), the CPU 200 and/or the memory 201 may be 
incorporated within the system ASIC 210, rather than as separate modules as illustrated in 
Figure 2a. 

Embodiments of the home media server 110 may also be equipped with a DVD 
drive, CD player, CD Read- Write drive, recordable DVD drive (as described in greater 
detail below), and/or any other type of portable storage medium 235. In one embodiment, 
these devices may communicate with the home media server 1 10 via an AT Attachment 
Packet Interface ("ATAPI"), although the type of interface used is not pertinent to the 
underlying principles of the invention. 

Figure 2b illustrates a software architecture employed in one embodiment of the 
home media server 1 10. Different hardware architectures 280 may be used to support the 
software, including the hardware architecture illustrated in Figure 2a. A multitasldng, 
multithreaded operating system ("OS") 270 (e.g., Linux, UNK, Windows NT®) with real 

9 



wo 02/21841 



PCT/USOl/22002 



time streaming support is executed on the hardware 280. In one embodiment, certain 
proprietary customizations 292 are programmed in the media server OS 270 including, for 
example, real time services for streaming audio and video (real time OSs typically do not 
include these features). 

A media toolkit 260 executed within the home media server 1 10 provides an 
application programming interface ("APF') for the different media server 1 10 applications 
described herein as well as a set of media server 1 10 utilities. In one embodiment, a 
minimum quality of service ("QoS") is defined within the media toolkit 260. The media 
toolkit 1 10 may be comprised of several functional layers including a media stream 
abstraction layer; a media stream resource noanagement layer; a security/conditional access 
layer; and a transport/storage abstraction layer. 

In one embodiment, the home media server 1 10 is configured with support for the 
realtime transport protocol ("RTP") and the realtime streaming protocol ("RTSP") (see, e.g., 
RTP/RTSP module 251). Briefly, RTP is an IP protocol which supports real time 
transmission of voice and video. An RTP packet typically resides on top of a user datagram 
protocol ("UDF') and includes timestamping and synchronization information in its header 
for proper reassembly at the receiving end. RTSP is a well known protocol for streaming 
multimedia content over a network. It should be noted, however, that various other 
streaming protocols may be implemented while still complying with the underlying 
principles of the invention (e.g., Quicktime®, Windows Media, . . . etc). 
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RTP and RTSP were designed primarily for PC to PC communication over non- 
realtime networks. Accordingly, because one embodiment of the home media server 1 10 
operates over a realtime home media network 140 and communicates to devices other than 
PCs (e.g., media nodes 150, 151), certain optimizations 290 may be prograrmned within the 
RTP/RTSP component 251. These include, for example, support for multiple data streams 
between the home media server 110 and the various multimedia devices 150, 15 1 ; 160-166 
(e.g., one or more video streams and several audio streams) 

Apache HTTP server support 250 is also included in one embodiment of the home 
media server 110. Apache is an open-source HTTP server software product which may be 
executed on various modem desktop and server operating systems. Once again, certain 
media server customizations 290 may be included within the Apache component 250, 
further refining HTTP support for the home media network 140 environment. 

In one embodiment, a plurality of device servers 252 are executed on the home 
media server 110. These devices servers provide application support for each of the media 
nodes 150-151 and other devices 160-166 communicating with the home media server 110 
over the home media network 140. For example, in response to a "tune" command sent 
from a media node 150, a tuning device server will cause the home media server to tune to a 
broadcast channel specified by the tune command. • The tuning server may also include 
conditional access functionality (i.e., only allowing tuning to occur for channels to which the 
user has access rights). 
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Although described above as a "software" architecture, it should be noted that 
various elements illustrated in Figure 2b may also be implemented in firmware and/or 
hardware while still complying the underlying principles of the invention. 

Referring again to Figure 1, numerous digital and analog devices may be configured 
to communicate with the home media server 1 10 over the home media network 140. By 
way of example, and not limitation, these include personal computers 160, cameras or 
digital camcorders 161, printers 162, notebook computers 163, automotive audio/video 
systems 164, cell phones or personal digital assistants 165, standard telephones 165 
(including fax machines), home security systems (not shown); and/or home climate control 
systems (not shown). 

In one embodiment, complex multimedia and data processing such as tuning to 

selected channels, recording of specified programs/music, storing phone numbers and 

personal data, connecting to remote network sites, etc., is performed at the media server 110, 

rather than at the individual network devices 160-166. As such, these devices 160-166 may 

be manufactured relatively inexpensively. For example, multimedia nodes 150, 151 may be 

equipped with just enough processing power and memory to receive and play back a 

multimedia signal, with storage and control (e.g., tuning) functions offloaded to the home 

media server 1 10. Similarly, a telephone 166 may be designed with nothing more than a 

low power microcontroller coupled to an RF transmitter, with telephony functions and 

contact data storage provided by the home media server 1 10. In addition, because these 

network devices 160-166 do not require as much circuitry, they will be lighter than 

currently-existing devices, and will consume less power. 

12 



wo 02/21841 



PCT/USOl/22002 



In one embodiment, the primary communication medium over whicli the home 
media server 110 and the various devices 150-151, 160-166 communicate is wireless RF 
(e.g., via network module 240), with terrestrial transport connections such as Ethernet 
reserved for devices which are not within RF transmission range. Moreover, certain devices 
which require a substantial amount of home media network 140 bandwidth (e.g., high 
deQnition television 170), and/or devices which are in close proximity to the media server 
110 may be configured to communicate over teixestrial transports, depending on the 
requirements of the particular configuration. 

One Embodiment of a Multimedia Node Architecture 
Distributed multunedia nodes 150 and 151 illustrated in Figure 1 provide an 
interface to the home media network 140 for audio systems 170 (e.g., audio amplifiers and 
speakers) and/or video systems 171 (e.g., standard television sets, wide screen television 
sets, high definition television ("HDTV") sets, or any other device capable of displaying 
video). 

As illustrated in Figure 6, one particular embodiment of a multimedia node 
architecture (see, e.g., node 150 in Figure 1), is comprised of a network interface 605 
coupled to the multimedia node bus 610 (e.g., a PCI bus); a system ASIC 620, including 
MFEG-2 decode/graphics logic 630 (or other multimedia decode logic); a central processing 
unit 640 and memory 650; an audio processing unit 660; and/or an infrared port 670. 

The local central processing unit 640 and memory 650 execute programs and process 

data locally (i.e., at the multimedia nodes 150, 151). The network interface 605, which may 
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be an RF interface or a terrestrial interface (e.g., Ethernet), receives/transmits multimedia 
content and control data from/to the home media server 110 over the home media network 
140. The system ASIC 620 decodes and processes incoming multimedia content transmitted 
from the home media server 1 10 via MPEG-2 decode/graphics logic 630 (or other 
multimedia compression standard) to produce one or more video outputs 680 (e.g., an NTSC 
output, an HDTV output, . . . etc). In one embodiment, a separate audio processing unit 660 
produces both digital and analog audio outputs, 681 and 682, respectively. Moreover, one 
embodiment also includes a local mass storage device (not shown) for stodng certain 
multimedia content and/or data (e.g., frequently-requested content/data). 

In one embodiment, the local infrared interface 670 receives control commands from 
a remote control unit (e.g., unit 532 in Figure 5) operated by a user. As described below 
with reference to Figure 5, control.data/commands received through the infrared interface 
670 may subsequently be transmitted to the home media server 1 10 for processing. For 
example, in one embodiment, if a user selects a command to change to a live broadcast 
channel (e.g., the evening news), the command is fransmitted from multimedia node 15 1 to a 
home media server 1 10 tuning application, which fcffwards the command to a video module 
(e.g., CATV module 241 , satellite TV module 243). The tuner in the video module (see, 
e.g., tuner 910 in Figure 9) then tunes to the requested frequency and the new video signal 
is streamed from the home media server 1 10 to the multimedia node 151. Alternatively, or 
in addition, if the requested content was previously stored on the home media server 110 
(e.g., on mass storage device 230), then no broadcast tuning is necessary - the content is 
simply read from the storage device 230 and transmitted to the multimedia node 150. 

14 



wo 02/21841 



PCTAJSOl/22002 



Accordingly, as described in greater detail below, the amount of audio/video content which 
must be broadcast to the home media server 110 (i.e., over dedicated broadcast channels) is 
significantly reduced. 

One embodiment of a multimedia node 150 software architecture is illustrated in 
Figure 6b. Various hardware architectures 690 may be used to support the software 
architecture, including the architecture illustrated in Figure 6a. An operating system 691 
executed on hardware 690 (e.g., Windows '98, Linux, . . . etc) includes client customizations 
for optimizing communication over the home media network 140 (e.g., providing low level 
support for real time streaming of audio and video). 

A standard set of user interface components 694 included in one embodiment may be 
employed (e.g., by application developers) to generate unique interactive interfaces at each 
of the media nodes 150, 151. For example, a user-navigable tuning index may be included 
which lists available content by dates/times and allows users to graphically select a 
particular broadcast channel and/or stored content from the mass storage device 230. 

In addition, support for the hypertext markup language ("HTML") and/or the 
extensible markup language ("XML") are included in one multimedia node 150 
embodiment, allowing users to download, view and modify various types of Internet content 
(e.g., Web pages). 

Applications executed on multimedia nodes 150, 151 may include client-specific 
applications 697 and/or home media server 1 10 control applications 696. Two examples of 
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media server control application 696 will now be described with reference to Figures 6c-6d. 
It should be noted, however, that these examples are for the purpose of illustration only and 
should not be read to limit the scope of the invention. 

Figure 6c illustrates an embodiment for tuning to a particular broadcast channel 
(e.g., cable, satellite broadcast). A user initially selects a particular channel (e.g., via a 
remote control device) to be viewed at the local multimedia node 151. The command is 
received by the local TV tuning application 601 executed on the multimedia node 151, 
which sends a tuning request to the home media server 1 10 over the home media network 
140. A tuner server module 602 executed on the home media server 110 receives the 
request and forwards the request to the tuner driver 271 (e.g., through the media toolkit 260). 
In response to the command, the tuner driver 271 directs the tuner hardware (see, e.g., tuner 
910 in Figure 9 described below) to tune to the specified channel. The tuner driver also 
communicates with the conditional access subsystem (see, e.g., module 930 in Figure 9) to 
detarrnine whether the user has the right to view the particular channel (e.g., the channel 
may be a subscription-based channel such as HBO). 

If the user has access rights, then the tuner server module 252 creates video stream 
for the specified broadcast channel and feeds the stream to the multimedia node 151. In one 
embodiment, the tuner server module 602 creates the stream using the RTP/RTSP protocol 
(i.e., by communicating with the RTP/RTSP module 251). The media node receives the 
stream through the local TV tuning application 601 and decodes the stream using a codec 



16 



wo 02/21841 



PCT/USOl/22002 



embedded within the media toolkit 260 (various additional encoding features of the system 
are set forth below). Once decoded, the video image is generated on the video display. 

Figure 6d Olustrates an embodiment for employing a music jukebox at a multimedia 
node 150. A user initially selects a particular music play list (e.g., via a remote control 
device) to be played at the multimedia node 150. The request is received by a music 
jukebox application 601 executed on the multimedia node 150, which forwards the request 
to the home media server 1 10. A jukebox server module 604 executed on the home media 
server 110 receives the request and forwards the request to the media server file system 606 
(through the media toolkit 260), which retrieves the music files from the mass storage device 
230. The jukebox server module 604 then creates the audio stream for the specified files 
and feeds the stream to the multimedia node 150. In one embodiment, the toner server 
module 252 uses the RTP/RTSP protocol to create the stream (e.g., through communication . 
with the RTP/RTSP module 251). The multimedia node 150 receives the stream through its 
jukebox application 601 and decodes the stream using a codec embedded within the media 
toolkit 260. Once decoded, the music tracks are reproduced on a set of speakers. 

Network Operations Center 

In one embodiment, a network operation center ("NOC") 180 illustrated in Figure 1 
provides data and multimedia services for the home media server 110. The NOC may be 
comprised of one or more servers communicating with the home media server 1 10 over the 
Internet 120 (or other network). In one embodiment, the NOC performs one or more of the 
following functions: 
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Remote Monitoring and Maintenance . The NOC 180 monitors the home media 
server to ensure that it is operating within normal parameters. In one embodiment, the home 
media server 110 periodically transmits a status update to the NOC 180, indicating whether 
any particular services are required and/or whether any problems have arisen. If the home 
media server 110 fails to transmit a status update after a predetermined period of time, the 
NOC 180 may take steps to determine the cause of the problem and/or take steps to contact 
the user. For example, diagnostic tests may be performed to evaluate the network 
connection between the home media server 1 10 and the NOC 180. If the diagnostic tests do 
not provide an answer, then the user may be contacted via telephone to determine the next 
course of action. In one embodiment the NOC 180 is staffed by engineers or other technical 
assistance personnel 24-hours a day. 

In one embodiment, the user's home security system and/or fire system 
communicates with the home media server 110 over the home media network 140. 
Accordingly, the home media server 1 10 may be programmed to relay home security and/or 
fire system data to the NOC 180 on a periodic basis during certain periods of tune (e.g., 
every minute). If a security problem or fire trouble indication is encountered during this 
period of time (e.g., a door/window is opened without the proper security access code), the 
NOC 180 may notify a local law enforcement agency and/or the local fire department. 

Automatic Downloading and Upigrades. In one embodiment, the NOC 180 provides 

automated software downloading and upgrades to the home media server 1 10. For example, 

when a new software interface/program (e.g., a new graphical user interface) is developed 

for the home media server 110 it may be automatically downloaded from the NOC 180 and 
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installed. Whether the NOC 180 should upgrade software on the home media server 110 
automatically may be determined by each individual user. Certain users may opt to 
manually upgrade all of their software while other users may specify a particular subset of 
software to be automatically upgraded. For example, a user may specify automatic upgrades 
for graphical user interface software and manual upgrades for email clients and word 
processors. 

hi addition, the NOC 180 may store up-to-date drivers for various home media 
network devices 160-166. In one embodiment, when a user purchases a new device, such as 
a new telephone, the device may automatically identify itself to the home media server 110 
(e.g., via USB, IEEE 1394 or other device identification technology). The home media 
server 1 10 will then determine whether the device is supported by the system and, if so, 
query the NOC 180 for the latest driver. In one embodiment, once the driver is identified, it 
is automatically downloaded ftom the NOC 180 and installed on the home media server 110. 
Once installed, the driver may be updated automatically as new updates become available. 
Accordingly, driver software will no longer need to be bundled with home network devices 
160-166, resulting an additional cost savings to the end-user. In addition, users will not be 
required to install and configure device drivers for each new network device they purchase. 

In addition, the home media server 1 10 may check the NOC 180 to determine 
whether the home network device has been certified by the NOC 180 as meeting some 
minimum level of quality and/or performance. In one embodiment, device drivers will be 
downloaded and installed on the system only if the device has been certified. In this 
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maimer, a minimum quality of service may be maintained for all devices employed on the 
system. 

The same type of device identification, certijHcation, and driver installation may be 
performed for communication modules 240-245 such as those described above. Thus, as 
with home network devices 160-166, certain communication modules 240-245 which do not 
meet a minimum quality of service requirenaent (e.g., those which perform inefficient bus 
transactions which tie up the system bus 220) will not be certified by the NOC 180 and will 
not be pramitted to be installed in the home media server 110. 

In addition, in one embodiment the communication modules 240-245 are 
manufactured and certified by third party content providers (e.g., satellite broadcast 
operators). In this embodiment, the content providers may configure the modules 240-245 
to work with their own proprietary communication protocols, encryption techniques, and/or 
application programs. For example, DirecTV™ may develop a proprietary interface for its 
users so that when a user selects his DirecTV feed he will be presented with a unique user 
interface and/or channel programming environment. Accordingly, while the present 
invention provides a standard media transport interface for a variety of different 
communication channels, service providers can stiU distinguish their services based on the 
proprietary applications/transports which they develop. 

Logging/Data Warehousing. In one embodiment, the NOC 180 may perform 

logging and data warehousing for the home media server 110. More specifically, the NOC 

1 80 may maintain a log of network transactions for each home media server 110 and 
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subsequently evaluate the log for a variety of reasons (e.g., to troubleshoot system problems, 
to deterniine a user's preferences and tailor services and/ca* advertising to that user, . . . etc). 
For example, by monitoring usage patterns, the NOC 180 may determine that every time a 
certain Java applet is downloaded, the home media server 110 crashes. As such, the NOC 
180 may takes steps to ensure that the applet in question is no longer downloaded by the 
home media server 110 (e.g., by notifying the user or automatically blocking the applet). 
The NOC 180 could then notify the technical support staff to determine the problem with the 
applet. 

Similarly, the usage log may be evaluated to determine the preferences of a user and 
to provide specialized services to that user based on those preferences. For example, based 
on the Web sites the user visits and/or the channels that the user watches, the NOC 180 may 
determine that the user is interested in baseball. As such, the NOC 180 may automatically 
provide baseball-related content to the user such as, for example, broadcast schedules for 
upcoming games, a subscription offer to a sports magazine, advertisements, and various 
other baseball-related content. Similarly, the NOC 180 may determine that the user watches 
certain television shows on a regular basis, and may automatically download/record those 
shows on the home media server 1 10 (e.g., via TCP/IP), so that they will be readily available 
for the user (e.g., during non-broadcast periods of time). 

Archiving . Li one embodiment, users may backup multimedia content and other 

types of data at the NOC 180. For example, a user may take a series of pictures with a 

digital camera and transmit the originals to the NOC 180 for developing. In one 

embodiment, the NOC wiD transmit the pictures to a developer on behalf of the user and will 
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Store a backup copy of each of the originals (the NOC 180 will be backed up regularly to 
prevent loss of the originals). 

In one embodiment, the NOC 180 will monitor all multimedia content purchased by 
the user over the Internet. For example, when the user downloads a new compact disk 
("CD") from a music download site, the NOC 180 will record the transaction in the user's 
profile. As such, the user does not need to store all of his/her multimedia content locally on 
the home media server 1 10. Rather, because the NOC 180 keeps track of all the content to 
which the user has access rights, the user can offload storage to the NOC 180 and re- 
download the content when necessary (e.g., following a hard drive failure on the home 
media server 110). 

Registration/confjeinration . In one embodiment, users may be required to register 
with the NOC 1 80 upon purchasing a home media server 110 and related network devices 
150-151, 160-166. The NOC 180 may prompt each user to respond to a series of question 
directed to the user's preferences, the type/level of NOC 180 services desired by the user, 
the authorization level of each user in the home media server 1 10 household (e.g., children 
may be provided with limited limited functionality), and any other user-related data which 
may aid the NOC 180 in providing user services. In one embodiment, the NOC 180 will 
automatically detect the home media server 1 10 configuration and store this data in a user 
database (e.g., to be used for software upgrades, troubleshooting, . . . etc). 

Bridge to Satellite/Cable Operations . In one embodiment, the NOC 180 will 

coordinate communication between the home media server 1 10 and any satellite/cable 
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services to which the user has subscribed. For example, the NOC 180 may forward pay-per- 
view requests transmitted from the home media server 1 10 to the various cable/satellite 
operators, and perform the back-end processing (e.g., authentication, billing) required for the 
pay-per-view transaction. Because a persistent communication channel exists between the 
NOC 180 and the home media server 1 10 (e.g,, through DSL or cable modem), no dial-up 
processing is required (unlike current pay-per-view cable and satellite systems). 

E-commerce Support. In one embodiment, rather than acting merely as a conduit 
between the home media server 110 and other e-commerce Internet sites, the NOC 180 may 
perform various e-commerce functions itself. For example, the NOC 180 may market and 
sell products on behalf of other online retailers (e.g., Amazon.com®). The NOC 1 80 of this 
embodiment may perform the back-end processing (e.g., billing and record keeping) 
required for each e-conam^ce transaction. 

Application Support . In one embodiment, the NOC 180 will provide support for 
applications executed on the home media server 110. For example, the NOC 180 may 
provide a compact disk database containing CD tides, track information, CD serial numbers, 
etc. When a user copies his CDs onto the mass storage device 230 of the home media server 
1 10, the home media server 1 10 may query the database (as described in greater detail 
below) and download tide and track information (or other information) used to 
identify/index each CD and each track. In one embodiment, public CD databases (e.g., such 
as the database maintained at "www.cddb.com") may be filtered and improved by the NOC 
180 to remove errors. Various other types of application support may be implemented at the 
NOC 180 consistent with the underlying principles of the invention. 
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Remote-Access Gateway . In one embodiment, the NOC 180 may be used as a 
gateway to access to the home media server 1 10 from a remote location. For example, a 
user from a PC connected to the Internet may log in to his home media server 1 10 through 
the NOC 180. 

In one embodiment, the home media server 1 10 is configured to make outgoing 
connections only (i.e., to reject direct incoming connections). As described briefly above, 
the home media server 110 may poll the NOC 180 periodically (e.g., every few seconds) and 
transmit a status update. During this periodic poU the home media server 1 10 may query the 
NOC 180 to determine whether anyone is attempting to access the home media server 1 10 
from a remote location. If so, then the NOC 180 transmits specific information related to 
the connection attempt to the home media server 1 10 (e.g., authentication data such as user 
ID'S, passwords, . . . etc). The home media server 110 may then initiate a connection with 
the remote user (e.g., using the remote user's IP address and encryption key). 

In one particular embodiment, the NOC 180 will authenticate the remote user before 
notifying the home media server 1 10 of the connection attempt. For example, upon receipt 
of a remote connection request, the NOC 1 80 may prompt the user to answer a series of 
questions (e.g., personal questions, questions related to the user's account, . . . etc). If the 
usra* does not answer the questions in a satisfactory manner, a member of the NOC staff may 
contact the ns&c directly (e.g., via standard telephone or IP telephone). Regardless of how 
authentication takes place, once a remote user is authenticated, the user's data is transmitted 
to the home media server 110, which subsequently establishes a connection with the remote 
user. 
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Additional Home Media Server Embodiments and Applications 
An external illustration of one embodiment of the home media server 110 is shown 
in Figures 3 and 4 (from the back and front, respectfully). Figure 3 shows a home media 
server 1 10 with a plurality of different media modules 320-325 installed in its expansion 
bays (i.e., electrically coupled to its system bus 220). As illustrated, in one embodiment, 
certain standard communication ports such as Ethernet 301, IEEE 1394 302, USB 303, 
digital/analog audio 305, standard telephone 304, XGA/HDTV 306, and/or other standard 
audio/video ports (e.g., AV output ports 307 and 308) may be installed directly in the 
primary home media server, rather than as an expansion card. 

As illustrated in Figure 4, one embodiment of the system includes a DVD player 420 
(and/or a CD player, C3D-ReWritable drive, recordable DVD drive or other type of portable 
digital media player/recorder). The DVD player 420 is capable of playing a DVD directly 
and/or transferring multimedia content from the DVD to the mass storage device 230. In 
one embodiment, as multimedia content is transferred to the storage device 230, either from 
the DVD player 420 or over the Internet 120, an indexing module executed on the home 
media server 1 10 indexes the multimedia content in a content database (not shown). 

Various DVD/CD identification techniques may be used to identify the particular 
DVD/CD inserted and copied to the storage device 230. For example, a checksum may be 
calculated for a known unique portion of the DVD/CD and compared with a CD/DVD 
checksum database (e.g., maintained at the NOC 180 or other server). Similarly, the serial 
number may be read from the DVD/CD and compared with a database of DVD/CD serial 
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numbers. Additional DVD/CD identification techniques which may be utilized in 
accordance with the principles of the invention are disclosed in co-pending applications 
entitled SYSTEM AND METHOD FOR SCALING A VIDEO SIGNAL, Serial No. 
09/632,458, filed August 4, 2000; and COMPACT DISK AND DIGITAL VIDEO DISK 

STORAGE AND PLAYBACK SYSTEM, Serial No. , Filed , each of which 

are assigned to the assignee of the present application and which are incorporated herein by 
reference. 

In one embodiment of the system, the storage device 230 (e.g., a hard drive) is 
preferably large enough to state hundreds of hours of video and/or audio content, as well as 
a variety of other digital information such as telephone voice messages, computer 
programs/data . . . etc. The current recommended size for the storage device 230 is at least 
80 gigabytes, however the particular size of die storage device is not pertinent to the 
underlying principles of the invention. 

One or more RF transmitters 430 are also provided in one embodiment of the home 
media server 110. The transmitter 430 (as well as the LAN 390, if one is installed) allows 
the home media server 1 10 to simultaneously transmit multimedia content and other types of 
data to the various media devices 150-151, 160-166 over the home media network 140 (e.g., 
at least one MPEG-2 video stream and several audio streams). 

In one embodiment, the wireless transmission system is capable of handling 

isochronous multimedia trafBc reliably (i.e., without jitter) in a wide range of residential 

settings. For example, the system must be enable of working in the presence of common 

26 



wo 02/21841 



PCT/USOl/22002 



residential RF interference such as microwave ovens and cell phones. In one embodiment, 
these requirements are met through the proper choice of modulation and coding. More 
specifically, one embodiment employs a modified version of IEEE 802.1 lb adapted to 
operate in a real time environment (e.g., using Reed-Solomon forward error-correction and 
antenna diversity with circular polarization). 

Figure 5 illustrates a home naedia server 1 10 communicating simtiltaneously with a 
video node 520 and an audio node 522. Certain multimedia nodes, such as node 522, 
include an LCD 524 (or other type of display) for displaying infcamation about the 
multimedia content stored on home media server 110 (e.g., CD and movie titles, CD tracks . 
. . etc). In one embodiment, the audio node 522 may be electrically coupled to an amplifier 
for amplifying the transmitted audio signal into a pair of speakers. As described above, in 
one embodiment, the individual multimedia nodes also include local storage devices (not 
shown) for locally caching recently used multimedia content and data. 

Data/commands sent from remote control devices 530, 532 are transmitted through 
the nodes 520, 522 to the home/media server 1 10 as indicated. Li one embodiment, the 
remote control devices include an LCD (not shown) or other type of display for displaying 
information about the multimedia content stored on home media server 1 10 (i.e., instead of, 
or in addition to, the multimedia node 522 display 524). 

As described briefly above, using the foregoing system, all of a user's data, music 

and video may be stared in a single location (i.e., home media server 110) and accessed 

firom anywhere in the house (e.g., stereo node 522) or the car (e.g., via an automotive 
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multimedia node 164 as indicated in Figure 1). Moreover, if the home media server HO is 
connected to the Internet through, for example, a persistent DSL connection 360, the user 
can access all of the stored content from various other locations across the globe (e.g., a 
sunmier home or a hotel while away on business). One embodiment of the system provides 
a secure, encrypted data stream when content/data is requested from the home media server 
1 10 in this manner, thereby protecting the user's privacy as well as the copyrights to the 
underlying multimedia content. 

If a multimedia node is employed in a user's automobile as described above, when 
the automobile is parked within range of the media server's RF transmissions, music or 
other audio/Addeo content may be transmitted and stored on a storage device within the 
automobile, even when it is not running. The specific type of audio/video contamt to be 
transmitted at a particular time during the day or week may be variable, based on users' 
preferences. For example, the home media server 1 10 may be configured to transmit up-to- 
date traffic reports during the morning and evening hours before the user leaves home/work. 
Hie user wiU then receive an instant trafBc report as soon as he starts his car in the morning 
and/or evening, followed by music or video sdected based on the user's preferences. 
Various additional aspects of this feature are set forth in the co-pending applications 
incorporated by reference above. 

In an embodiment which includes a CD-ReWritable ("OD-RW") drive, users can 
produce custom CDs using the audio content stored on the mass storage device 230 or 
downloaded from the Internet. The CD-RW drive may utilize serial copy management 

techniques to produce CDs which cannot themselves be copied, thereby protecting the 
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copyrights to the underlying works. Moreover, in one embodiment, the system will 
determine whether the content owner or content creator allows copying of the multimedia 
content. For example, certain content creators/owners may allow a copy to be stored on the 
home media server mass storage device 230, but may not allow copying to a portable disk 
(e.g., a CD-RW disk). In one embodiment, content stored on the mass storage device 230 
may be labeled as non-copyable by setting one or more "copy" bits to a particular value 
(e.g., in a bit field associated with the content). 

In one embodiment of the system, a portable musicArideo player node may be 
configured to retrieve multimedia content direcfly fi:om the home media server 110 or via an 
adapter module. Portable MPS players such as the Rio™ from Diamond Multimedia™, for 
example, may be used to store and play back digital audio content transmitted from the 
home media server in a flash memcxry module. In one particular embodiment, the CD-RW 
drive 420 is capable of accepting the 1/3 inch "mini CD" format. Accordingly, users may 
produce unique mini CDs (e.g., using fracks stored on the mass storage device 230), for 
playback on MP-3 players enable of playing mini-CDs. 

MPEG conversion logic is configured on one embodiment of the home media server 

110 (e.g., in software, hardware or any combination thereof), allowing the home media 

server 110 to convert from one MPEG format to another. Using the MPEG conversion 

logic, video content stored in MPEG-2 format may be converted to MPEG-1 format and 

recorded on a 'Video CD" (a compact disc format used to hold full-motion MPEG-1 video). 

As such, users may store video on a portable medium, even on a home media server 1 10 

which is not equipped with a recordable DVD drive. 
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In another embodiment of the system, home appliances (e.g., the refrigerator, the 
toaster, the air conditioner) and other home systems (e.g., security, air conditioning) are all 
provided with RF transmission devices to commraiicate with the home media server 1 10. 
Each device may also be configured with it's own intranal network address and/or Internet 
address. Users may then access information pertaining to these devices and/or control these 
devices from any room in the home or over the Internet. In one particular embodiment, the 
user's automobile is outfitted with an RF transmitter and a network address. Thus, in this 
embodiment, the automobile is capable of reporting maintenance information to the user via 
the home media server 110 (e.g., low brake pads, oil change needed . . . etc). 

In one embodiment, a user may publish a home Web page containing up-to-date 
information on each home appliance or other network device. Accordingly, users of this 
embodiment are able to monitor and control home appliances and systems £rom anjrwhere in 
the world. Li one embodiment this includes the ability to select broadcast listings and direct 
the home media server 110 to make recordings (e.g., based on date/time or broadcast ID 
code). In addition, in one embodiment, users may connect remotely to the home media 
server to review email and/or voicemail listings (which may be displayed to the user in the 
form of a single, generic "message" box). Voicemail messages may be streamed to the 
remote user's location over the Internet or other network. 

Telephony Embodiments 
As illustrated in Figure 7a, in one embodiment, home telephone devices 166 may 
also be configured to run through the home media server 1 10. In this embodiment, 

incoming faxes and voicemail are stored on the home media storage device 230 and may be 
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accessed from any room in the house (e.g. from telephone devices 166, personal computers 
160, PDAs 165, and/or video systems 151/171). In addition, phone number and address 
information may be stored in a contact database 740 on the home media server 1 10 and 
accessed through the various telephone devices 166 (or other home media devices). 
Offloading user contact data from the telephone devices 166 in this manner allows telephone 
devices 166 to be manufactured will less memory and less processing power, further 
decreasing costs to the end user. 

As illustrated in Figure 7a, in one embodiment, the home media server 110 includes 
a telephone connection module 730 which coordinates between standard telephony calls 
placed/received over the public switched telephone network ("PSTN") 170 and calls placed 
over the Internet 120 using IP telephony protocols. The telephone connection module 730 
will automatically route incoming calls from both sources to the same set of home telephone 
devices 166 (or other home media devices such as the user's personal computer 160). 

Users may specify whether a particular outgoing call should be placed over the 

Internet 120 (e.g., to an IP telephony device 710) or over the PSTN 170 (e.g., to a standard 

telephone device 720). In one embodiment, the telephone connection module 730 analyzes 

each outgoing telephone coimection request to determine whether the call should be routed 

through the Internet 120 or through the PSTN 170. The telephone connection module 370 

may factor in various types of connection data to make telephone connection determinations. 

For example, a user may specify certain contacts within the contact database 740 for which 

IP telephone connections should be used and certain contacts for which standard telephone 

connections should be used. In one embodiment, the telephone coimection module 730 will 

31 



wo 02/21841 



PCT/USOl/22002 



select a particular connection based on whether it is the least expensive option for the user 
(e.g., for contacts with both IP and standard telephone capabilities). Another variable which 
may be factored into the selection process is the connection throughput available to the user 
over the Internet. More specifically, the telephone connection module 730 may initially test 
the connection throughput (including the availability of a reliable connection) over the 
Internet 120 and place an IP telephony call only if some nodnimum level of 
throughout/reliability is available. Any of the foregoing variables, alone or in combination, 
may be evaluated by the telephony connection module 730 to select and appropriate 
telephone connection. 

One embodixoent of a protocol architecture for supporting IP telephony and related 
communication functions on the home media server 1 10 is illustrated in Figure 2b. Various 
telecommunication standards are supported by this embodiment, including the H.248 media . 
gateway control standard ("MEGACO") standard; the ITU-T H.323 and session initiation 
protocol ("SIP") standards for multimedia videoconferencing on packet-switched networks; 
the Realtime Control Protocol ("RTCF') standard - a conipanion protocol to the realtime 
transport protocol ("RTP") (described above) used to maintain a minimum QoS; and the 
ITU T.120 standard for realtime data conferencing (sharing data among multiple users). 

In addition, various audio and video codecs are supported by the illustrated 
embodiment, including G.711, G.723, and G.729 (for audio); and H.261 and H.261 (for 
video). As illustrated, each of these codecs may be executed above RTP, an IP protocol that 
supports realtincie transmission of voice and video. Each of the foregoing IP communication 
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protocols may be utilized by media server 1 10 application programs through the 
programming interfaces of the media toolkit 260. 

It should be noted that, although the embodiment illustrated in Figure 7b includes a 
specific set of communication protocols, various other communication protocols may be 
implemented within the home media server 110 while still complying with the imderlying 
principles of the invention. For example, in one embodiment, new protocol stacks (both 
proprietary and industry-standard) may be automatically transmitted from the NOC 180 and 
installed on the home media server 110 as they become available. 

Figure 7c illustrates a signal diagram representing the set up and termination of a 
telephony connection according to one embodiment of the home media server 1 10. At 750, 
the home media server transmits a call connection "invitation" on behalf of User 1 to User 2. 
Once User 2 accepts the call, a "success" message is transmitted back to the home media 
server 1 10 at 751. Ja response, the home media server 110 acknowledges receipt of the 
"success" message at 752 and aUocates a media stream 760 to support bi-direction audio 
communication between User 1 and User 2. To tear down the media stream 760, one of the 
users must hang up the phone, resulting in a "bye" message 753, 754 followed by a 
termination acknowledgement ("success") from both sides 755, 756. 

Broadcast and Packet-Switched Channel Coordination 

In one embodiment illustrated in Figure 8b, content providers 830 may transmit 

content to home media server 1 10 over Internet channels 120 as well as standard broadcast 

channels 170. The transmitted content may then be cached locally in a content database 
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850. When a user subsequently requests the same content to be played back on an audio 
device or video device 171 (e.g., via media node 151), it will be replayed to the user directly 
from the local content database rather than over a broadcast channel. In one embodiment, a 
content selection module 840 will determine whether to retrieve the content through a 
broadcast channel or directly from the content database 850. 

In one embodiment, the home media server 1 10 and/or the content provider 830 will 
monitor the preferences of each user in the media server 110 household to determine the 
content which will be requested during certain times of the day, and/or days of the week. 
For example, if a user watches "the Simpsons®" at 6:00 PM, the home media server 1 10 
and/or content provider 830 will record this behavior. The content provider 830 may then 
transmit the latest Simpsons episode to the home media server 1 10 over the Internet 120 
before its scheduled broadcast at 6:00 PM (e.g., during periods of the day or evening when 
traffic on the Internet 120 is low). Various other pre-recorded broadcast programs may be 
transmitted to the home media server 1 10 over the Internet rather than over a dedicated 
broadcast channel, thereby freeing up a substantial amount of inefiiciently used broadcast 
network bandwidth (pre-recorded content currently represents the vast majority of all 
broadcast content). 

As indicated in Figure 8b, this embodiment of the home media servex 100 will 
provide a seamless mechanism for transitioning from standard digital broadcast channels 
(e.g., MPEG-2 channels) and analog broadcast channels to packet switched isochronous 
channels (for live broadcasts) and asynchronous chaimels (for pre-recorded broadcasts). 
Initially, the breakdown between broadcast and packet switched chaimels (e.g., TCP/IP) will 
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be similar to that illustrated in column 804, with analog broadcast 803 taking up 
approximately half of the available bandwidth, with the other half split between digital 
broadcast 802 and packet switched channels 800. As content providere begin to switch over 
to a packet-switched transmission scheme, this breakdown will change as indicated at 
bandwidth allocation 805, with packet-switched channels taking up approximately half of 
the available bandwidth and the remainder split between analog broadcast 812 and digital 
broadcast 811. 

Finally, when standard analog and digital broadcast channels have been phased out 
completely, all content will be delivered over packet switched channels as indicated in 
bandwidth allocation 806. Many packet switched protocols support both asynchronous and 
isochronous data transmissions. Accordingly, several isochronous channels may be 
provided to support live, real time events (e.g., sports events, the evening news, . . . etc) and 
the remainder of tihe bandwidth may be allocated to pre-recorded events (e.g., sitcoms, game 
shown, talk shows, . . . etc). These pre-recorded events/shown may be asynchronously 
downloaded to the home media server 110 at any time of the day or evening (e.g., during 
periods of low network activity), resulting in a much more efficient allocation of available 
transmission bandwidth. 

Because embodiments of the home media server 110 described above are capable of 
receiving, storing and coordinating content transmitted from both packet switched channels 
(e.g., TCP/IP) and standard broadcast channels, the transition illustrated in Figure 8b may 
be accomplished seamlessly, while at the same time improving the overall quality of service 
to the end user. 

35 



wo 02/21841 



PCTAJSOl/22002 



In one embodiment, several concuirent, real-time multimedia streams are supported 
by the home media server 1 10 through buffering and/or disk storage techniques as illustrated 
in Figure 8c. More specifically, incoming multimedia content 860-862 from several 
different channels (e.g., standard broadcast channels 860, 861 and packet switched channels 
862, 863) may be buffered in a set of input buffers 870-873 and output bufifers 890-893 on 
the home media server 110. The input buffers and output buffers 870-873 and 890-893, 
respectively, may be portions of memory allocated within the main memory 201 (see Figure 
2). Alternatively, or in addition, the input/output buffers may be configured on the 
communication modules 240-245, within the system ASIC 210, and/or as separate modules 
on the home media server 110 motherboard. 

In one embodiment, the mass storage device 230 reads the multimedia data from 
each of the input buffers 870-873 and writes the data to a set of multimedia tracks 880-883. 
The mtdtimedia data is subsequently read fi:om each of the multimedia tracks 880-883 to a 
set of output buffers 890-893, from which it is transmitted to one or more of the destination 
multimedia devices 150, 151, 163. Buffering data streams on the home media server 1 10 in 
this manner provides significant cost efficiencies for the entire system. Specifically, 
because the multimedia data is buffered at a single distribution point within the home media 
network 140 (i.e., the home media server 1 10), no buffering is required at each of the 
individual multimedia devices 150, 151 and 160-166, thereby significantiy reducing the cost 
and complexity of these devices. 
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The mass storage device 230 of this embodiment has enough bandwidth in its 
read/write operations to support several multimedia streams at once. Although the seek time 
of the mass storage device 230 (i.e., the time required to jump from one track to another) 
may be relatively low (i.e., relative to its read/write bandwidth), buffering of the multimedia 
data in input and output buffers as described ensures that the various streams will be 
provided to their respective multimedia devices 150, 151 and 160-166 without intaruption. 

In one embodiment, each of the multimedia streams 860-863 operate independently 
of one another. For example, if playback of a particular multimedia stream is paused, e.g., 
from multimedia device 150, the home media server 110 wiD intemipt transmission of 
multimedia content from the output buffer 893 associated with that device 150, and wiU 
begin storing subsequent, incoming multimedia content on the mass storage device 230 
.(e.g., within the media track 883). However, this will not affect playback at any of the other 
multimedia devices on the network 140. When playback is resumed, the content will be 
streamed from the output buffer 893 (and media track 883) from the same point at which is 
was paused (thereby providing real time, time-shifting functionality for live, real-thne 
broadcasts). 

In one embodiment, an indexing module (not shown) will generate a content index 
for users of the home media server 1 10 which wiU provide users with a comprehensive, 
seamless listing of multimedia content from the Internet (e.g., streaming media content), 
broadcast channels (includmg live and pre-recorded broadcast channels), 
asynchronous/isochronous multimedia channels, and/or various other media 

sources/channels. Accordingly, this embodiment will provide users with the ability to 
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navigate through and select content from a virtually unlimited number of different 
multimedia sources and channels having any knowledge of the underlying protocols and 
communication infrastructure supporting the sources/channels. 

Intelligent Buffering and Bandwidth Allocation 
Virtually all communicatian channels are bandwidth-limited in some manner, due to 
the physical limitations of the underlying transmission medium and/or the signaling 
limitations of the channel (e.g., the channel's allocated frequency spectrum). For example, a 
100 Base-T Ethernet network is capable of providing a total data throughput of 100 Mbps, 
which is shared by all nodes (e.g., computers) on the network. Similarly, multimedia 
devices 150, 151; 160-166 communicating on the home media network 140 described above 
all share some maximum network signaling rate (e.g., if a standard IEEE 802.11b network is 
employed the signaling rate is 11 Mbps). 

Because bandwidth is shared on these systems, overall network performance may 
degrade as a result of nodes performing concurrent, high-bandwidth network transactions. 
This can be particularly problematic on networks such as the home media network 140 
which support real-time multimedia services. More specifically, high quality audio and 
video signals transmitted from the home media server 1 10 to multimedia nodes 150, 151, 
require some minimum level of network bandwidth to be rendered properly (i.e., without 
distortion or intearuption). Accordingly, one embodiment of the invention includes 
intelligent buffering and bandwidth allocation techniques to ensure smooth playback of the 
multimedia signal at each of the multimedia nodes 150, 151. 
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Figure 9a illustrates a histogram of normalized bitrate data for a particular 
multimedia stream (e.g., a DVD) over a period of 120 seconds. Curve 902 shows the bitrate 
averaged at one second intervals whereas curve 904 shows the bitrate averaged at 10 second 
intervals. The average bitrate over the entire 120 second interval is represented by curve 
906 (approximately 4.7 Mbps). Accordingly, based on the histogram data, the effective 
bitrate required to properly render the video stream at multimedia node 150 fluctuates 
significantly - from a maximum throughput of 7.143 Mbps to a minimum throughput of 
1.657 Mbps. 

One embodiment of the home media server 110, illustrated in Figure 9b, uses this 
type of bitrate data to allocate bandwidth and buffering levels for multimedia playback. 
Accordingly, when a user selects audio or video content (e.g., a DVD) to be transmitted to a 
multimedia node (e.g., video node 151), the home media server 1 10 initially retrieves a 
bitrate template 930 (e.g., a histogram of bitrate data) associated with the requested content 
from a bitrate database 935. The database 935 may be maintained locally on the home 
media server 1 10 and/or, as illustrated, may be maintained on a remote database server 940 
(e.g., maintained at the NOC 180). In either case, the database 935 may be continually 
updated as new multimedia content becomes available. Once the bitrate template 930 is 
initially downloaded from the remote server 940, a copy may be stored locally on the home 
media server 1 10 for future reference. In one particular embodiment, the bitrate template 
930 may be transmitted along with the underlying multimedia signal (e.g., embedded within 
the MPEG-2 stream read from a DVD or broadcast over a cable channel), rather than 
maintained in a separate database 935. 
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If the bitrate template 930 is maintained in a database 935, various types of 
identification data 925 may be used to identify the correct template for the multimedia 
content being played including, but not limited to, those set forth in co-pending application 
entitled SYSTEM AND METHOD FOR SCALING A VIDEO SIGNAL, Serial No. 
09/632,458 (incorporated by reference above). For example, a checksum may be calculated 
for a known unique portion of the multimedia content and compared with checksums stored 
in the database 935. Similarly, if the content is stored on a CD or DVD, the CD/DVD serial 
number (or other embedded identification code) may be used to perform a database 935 
query. Various other identification techniques may be used to identify the multimedia 
content while still complying with the underlying principles of the invention. Once the 
correct bitrate template 930 is identified, it is transmitted from the remote server 940 to the 
home media server 110. Alternatively, if the data is stored locally on the home media server 
1 10, it is retrieved directly from the home media server' s storage device 230. 

In one embodiment, an allocation module 950 running on the home media server 110 
analyzes the bitrate template 930 to establish an efficient bandwidth allocation and/or 
buffering policy for transmitting the multimedia stream(s). The allocation module 950 acts 
as a data "throttle," increasing or decreasing the data throughput from the home media 
server 1 10 to the multimedia nodes 150-151 as necessary to meet the bitrate requirements of 
each multimedia stream (e.g., through control of the home media server output buffers 910- 
912 and/or RF transceiver 915). The goals of one embodiment of the system are (1) to 
ensure that the underlying multimedia content is reproduced at the multimedia node 151 
without interruption/jitter; (2) to minimize the memory requirements at the multimedia node 
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151; (3) to minimize the playback delay experienced by the end user; and/or (4) to minimize 
the bandwidth required to accurately reproduce the multimedia content at the node 151. 
Any of these goals, alone or in combination, may be factored into the allocation module's 
150's bandwidth/buffering allocation policy. 

In one embodiment, the allocation module 950 analyzes the bitrate template 930 to 
ensure that the amount of multimedia content in each multimedia node buffer - e.g., buffer 
920 of multimedia node 151 - is sufficient to handle upcoming spikes. For example, given 
the bitrate template data set forth in Figure 9c, the multimedia node buffer 920 must have 
sufficient multimedia data (i.e., an adequate number of bits) to handle the bitrate spike 
between 3 seconds and 4 seconds (i.e., 6.2 Mbps). As such, by analyzing the bitrate 
template 930 as a whole, the allocation module 950 may increase the allocated data 
throughput between 1 second and 3 seconds to sufficiently fill the buffer 920 by the time the 
bitrate spike arrives (i.e., at 3 seconds). Hius, the number of bits consumed during the spike 
(6.2 Meg) must be less than or equal to the number of bits in the buffer at start of spike 
period (3 seconds) minus the bits added to buffer during spike period {i.e., the per-second 
bitrate). Oflierwise, playback of the multimedia stream wiU stall due to an ondetrun 
condition (i.e., a lack of multimedia data at the multimedia node 151). 

The allocation module 950 may also factor in bitrate templates 930 of other 
multimedia streams when making allocation decisions for a given stream. For example, 
when making allocation decisions for the DVD stream in Figure 9b (which passes through 
buffers 910 and 920), the allocation module 950 may evaluate the bitrate requirements of the 
other two streams (i.e., the MP3 stiream and Cable sti^am passing through buffers 911, 921 
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and 912, 922, respectively). As such, if one particular stream requires a significant amount 
of throughput at a given point in time, the allocation module wiU take anticipatory steps to 
ensure that sufficient multimedia data will be available. For example, in Figure 9d, the 
bitrate data illustrated in timeline 960 indicates a severe bitrate spike between 40 and 60 
seconds. The spike is severe enough that in order to properly render the multimedia stream 
associated with it, the allocation module 960 may need to fill the multimedia node buffer by 
the time the spike arrives (i.e., at 40 seconds) and also divert a significant amount of the 
system bandwidth to the stream to avoid an underrun condition. As such, in anticipation of 
the spike, the allocation module 950 may attempt to fill all the buffers on the system 
including the buffers of the other two streams (associated with the bitrate data shown in 
timelines 961 and 962) so that by the time the spike arrives, it can divert bandwidth from 
these streams to the stream with the bitrate spike. It should be noted that the foregoing 
bandwidth/buffer allocation example was for the purpose of illustration only and should not 
be read to limit the scope of the invention. 

In one embodiment, the when a new playback stream is requested at a multimedia 
node 151, the allocation module 950 will initially allocate all system bandwidth available at 
that time to the stream. It will subsequenfly decrease the bandwidth (i.e., the "throttle") only 
when other multimedia nodes require additional bandwidth and/or when the buffer at the 
multimedia node 151 fills up. Accordingly, one goal of this embodiment is to keep the 
buffer at the multimedia node 151 filled to its limit at all times (e.g., so that it will be ready 
for upcoming spikes). 
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Playback of the multimedia data from the multimedia node buffer 920 may start at 
different times, depending on the histogram data associated with the content. For example, 
the hitrate for certain high definition television broadcasts or DVDs such as the one 
illustrated in Figure 9e (which shows bitrate data for the movie "South Park") may start off 
at a relatively high value (i.e., approximately 8.6 Mbps). As such, it may be necessary to 
accumulate a certain amount of multimedia data in the multimedia node buffer 920 before 
initiating playback. The table 970 illustrated in Figure 9e shows the amount of data that 
must be buffered before playback (column 972), given a particular multimedia stream bitrate 
(column 971). The table 970 also indicates the amount of time which a user must wait 
before playback begins (column 973). As indicated in the bottom row of the table 970, at a 
bitrate of 9Mbps, ho buffering of the multimedia signal is required. In one embodiment, 
playback is initiated by the allocation module 950 (as indicated by the control signal 952 in 
Figure 9b). 

In contrast to multimedia stream described by the histogram of Figure 9e, the 
multimedia stream described by the histogram illustrated in Figure 9t (the DVD for the 
movie "Species 1") requires significantly less ba£fering before playback. More particularly, 
as indicated in table 975, no pre-playback buffeting is required at a bitrate of 5 Mbps. As 
such, if 5Mbps car more system bandwidth is available, playback may begin as soon as the 
stream reaches the multimedia node 151. In addition, in one embodiment, the allocation 
module 950 may initially allocate more bandwidth than required to fill the multimedia node 
buffer 920 during playback (e.g., so that more system bandwidth may be allocated elsewhere 
once the buffer is full). 
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In one embodiment, when a new playback stream is requested from a multimedia 
node 151, the allocation module 950 may initially aUocate just enough system bandwidth to 
meet some minimum playback delay threshold. Thus, if this minimum delay threshold is set 
at 4.5 seconds, the allocation module 650 must allocate at least 7 Mbps to reproduce the 
multimedia content represented by the histogram of Figure 9e (see Table 970, 5" row). By 
contrast, to meet the minimum delay threshold for the multimedia content represented by the 
histogram of Figure 9f, the allocation module 950 may allocate a bitrate of less than 5 Mbps 
(see Table 975). After the initial bandwidth allocation, the allocation module may use the 
histogram data to ensure that the amount of multimedia content in the multimedia node 
buffer 920 is sufficient to handle upcoming spikes in bitrate (as described above). 

Various additional bitrate histograms and related bandwidth allocation tables are set 
forth in Figures 9g through 9m. It should be noted, however, that these histograms are 
provided merely for the purpose of illustration, and should not be read to limit the scope of 
the present invention. Moreover, although the examples set forth above focus on the 
reproduction of DVD movies at multimedia nodes on a home media network, it shoidd be 
noted that the underlying principles of the invention may be implemented on virtually any 
system in which time-based data is transmitted over a bandwidth-limited network. 
Accordingly, the scope and spirit of the present invention should be judged in terms of the 
claims set forth below. 

Conditional Access and Encryption 
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One embodiment of a cable broadcast module for receiving one or more broadcast 
channels is illustrated in Figure 10. This embodiment includes two tuners 1010, 1011 for 
concurrently tuning to two separate broadcast charmels (e.g., to be viewed at two separate 
multimedia nodes on the home media network 140). Additional tuners may be included 
within the cable broadcast module as necessary (i.e., for receiving additional broadcast 
channels). 

In one embodiment of the system, a smartcard 330 (see also Figures 3) inserted into 
one or more of the media modules 320-325 may be programmed with a specific set of access 
rights defining the particular channels to which users are permitted access. The smartcard 
330 may work in conjunction with a conditional access module 1030, which will prevent 
certain channels/content firom being transmitted on the system which the user does not have 
the right to receive. In one embodiment, different members of a family may be assigned 
smartcards 330 with different access privileges. For example, children may be assigned 
smartcards 330 which limit the channels and/or Internet sites which they are permitted to 
access. 

In the illustrated embodiment, the signal outputs from each of the tuners 1010, 1011 
are fed into separate audio and video analog-to-digital decoder modules 1024, 1026 (audio) 
and 1023, 1025 (video), respectively. In one embodiment, the signals are then re-encoded 
with a compression codec. For example, in the embodiment shown in Figure 9, MPEG-2 
encoder modules 1031, 1033 encode the audio/video signals into MPEG-2 format (i.e., 
which may subsequently be decoded by any multimedia devices 150, 151 on the home 
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media network 140). It should be noted that various other digital compression codecs may 
be used (e.g., AC-3, MPEG-1, . . . etc) while still complying with the underlying principles 
of tiieinventicHi. 

In addition, in one embodiment, before being transmitted onto the system bus 220, 
the encoded multimedia signals are encrypted by an encryption module 1060 using a unique 
encryption key 1061. As such, in this embodiment, all multimedia data stored on the home 
media server 110 (i.e., on the mass storage device 230) and transmitted across the home 
media network 140 are encrypted, thereby preventing copying and playback by any 
unauthorized devices listening in one the network 140 or on the system bus 220. Moreover, 
authorized multimedia devices 150, 151, 160-166 (i.e., those certified by theNOC 180) 
communicating on the home media network 140 are configured with the same unique 
encryption key 1061, thereby allowing them to properly decode the multimedia data 
transmitted fiom the home media server 1 10. 

In one embodiment, the encryption key 1061 is transmitted securely to the home 
media server 110 and media devices 150, 151; 160-166 through the conditional access 
system. Alternatively, or in addition, the key 1061 may be transmitted using conventional 
secure communication techniques such as Secure Sockets Layer ("SSL") or Data Encryption 
Standard ("DES"). 

One embodiment of a satellite module for connecting the home media server 110 

over a satellite channel is illustrated in Figure 11, including multiple tuners 1 1 1 0, 1 1 1 1 ; 

forward error correction modules 1130, 1140; a smartcard 1120 and associated conditional 
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access module 1 150; and an encryption module 960 for encrypting content transmitted onto 
the system bus 220 (and across the home media network 140). The different decoding, 
encryption, and autiientication features described above with respect to the cable broadcast 
module illustrated in Figure 9 are also configured into one embodiment of the satellite 
module. 

One embodiment of a cable modem for connecting the home media server 1 10 over a 
cable channel is illustrated in Figure 12. The cable modem includes a modulator unit 910, a 
Data Over Cable Service Interface Specification ("DOCSIS") module 920, and a data 
pump/PCI interface unit 930. Various different types of cable modems may be coupled to 
the home media server 1 10 while stUl complying with the underlying principles of the 
invention. 

In addition to the encrypticm system described above, one embodiment of the system 
provides copyright protection through the use of copyright tags associated with content 
stored on the home media server 110. The tags of this embodiment identify which types of 
content/information may be copied or streamed to which multimedia devices. As illustrated 
in Figure 13, tags may be set to a variety of copy/playback levels, including, for example, a 
tag 1311 indicating no restrictions on copying for audio file 13 10; a tag 1321 allowing only 
a single serial copy (e.g., to a CD-RW disk) with no cc^ying to personal computers fear 
audio file 1320; a tag 1331 permitting playback to local TVs (i.e., within the home media 
network 140) for video file 1330; and a tag 1341 allowing 24-hour playback to local TVs 
only (e.g., similar to a pay-per-view feature) for video file 1340. Accordingly, this 
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embodiment provides an additional mechanism for providing secure storage and 
transmission of copyrightable material. In one particular embodiment, content providers 
(e.g., cable and satellite providers) are provided with the ability to program tags into their 
content as necessary. 

Embodiments of the present invention include various steps, which have been 
described above. The steps may be embodied in machine-executable instructions which 
may be used to cause a general-purpose or special-purpose processor to perform the steps. 
Alternatively, these steps may be performed by specific hardware components that contain 
hardwired logic for performing the steps, or by any combination of programmed computer 
components and custom hardware components. 

Elements of the present invention may also be provided as a computer program 
product which may include a machine-readable medium having stored thereon instructions 
which may be used to program a computer (or other electronic device) to perform a process. 
The machine-readable medium may include, but is not limited to, floppy diskettes, optical 
disks, CD-ROMs, and magneto-optical disks, ROMs, RAMs, EPROMs, EEPROMs, magnet 
or optical cards, propagation media or other type of media/machine-readable medium 
suitable for storing electronic instructions. For example, the present invention may be 
downloaded as a computer program product, wherein the program may be transferred from a 
remote computer (e.g., a server) to a requesting computer (e.g., a client) by way of data 
signals embodied in a carrier wave or other propagation medium via a communication link 
(e.g., a modem or network coimection). 
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Throughout the foregoing description, for the purposes of explanation, numerous 
specific details were set forth in order to provide a thorough understanding of the present 
system and method. It will be apparent, however, to one skilled in the art that the system 
and method may be practiced without some of these specific details. Accordingly, the scope 
and spirit of the invention should be judged in terms of the claims which follow. 
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CLAIMS 

What is claimed is: 

1 . A method comprising: 

receiving first multimedia content over a first communication channel; 

receiving second multimedia content over a second communication channel, said 
second communication chaimel being incompatible with said first cammunication channel; 
and 

distributing said first and second multimedia content to a plurality of multimedia 
nodes over a local multimedia network responsive to requests firom said multimedia nodes 
for said first and second multimedia content. 

2. The method as in claim 1 wherein said first communication channel is a 
broadcast channel. 

3. The method as in claim 2 wh^ein said second communication channel is a 
packet-switched channel. 

4. The method as in claim 3 wherein said packet-switched channel is a TCP/IP 
channel provided by digital subscriber line ("DSL"). 

5. The method as in claim 3 wherein said packet-switched channel is a TCP/IP 
channel provided by a cable modem connection. 

6. The method as in claim 2 wherein said broadcast channel is a broadcast cable 
channel. 

7. The method as in claim 6 wherein said second channel is a digital satellite 
channel. 

8. The method as in claim 1 wherein said local multimedia network is a wireless 
network. 
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9. The method as in claim 8 wherein said local multimedia network is a real-time 
network. 

10. The method as in claim 8 wherein said local multimedia network is comprised 
of a wireless portion and a terrestrial portion. 

11. The method as in claim 1 farther comprising: 

storing said first multimedia content on a stcn^ge device for subsequent distribution 
to said multimedia nodes. 

12. The method as in claim 1 further comprising: 
receiving audio over a public switched telephone network; and 
transmitting said audio to one or more of said multimedia nodes. 

13. The method as in claim 12 fijither comprising: 

storing said audio on a storage device for subsequent distribution to one or more of 
said multimedia nodes. 

14. A multimedia system comprising: 

a home media server configured to receive and store multimedia content transmitted 
from two or more incompatible conmiunication channels; and 

a plurality of multimedia nodes configured to communicate with said home media 
server over a real-time home media network, 

wherein said home media server transnodts said multimedia content to said one or 
more multimedia nodes responsive to requests for said multimedia content transmitted from 
said multimedia nodes. 

15. The system as in claim 14 wherein one of said coimnunication channel is a 
broadcast channel. 

16. The system as in claim 15 wherein another of said conomunication channels is a 
packet-switched channel. 
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17. The system as in claim 16 wherein said packet-switched channel is a TCP/DP 
channel provided by digital subscriber line ("DSL"). 

18. The system as in claim 16 wherein said packet-switched channel is a TCP/IP 
channel provided by a cable modem. 

19. The system as in claim 15 wherein said broadcast channel is a broadcast cable 
channel. 

20. The system as in claim 14 wherein said home media network is a wireless 
network. 

21. The system as in claim 14 wherein one of said multimedia chaimels is a public 
switched telephone network channel. 

22. A home media server comprising: 

means for receiving broadcast multimedia content; 

means for receiving packet-switched multimedia content; 

means for converting said broadcast multimedia content and said packet switched 
multimedia content into a standard multimedia content format; and 

means for distributing said multimedia content in said standard format to a plurality 
of multimedia nodes. 

23. The home media server as in claim 22 wherein said means for distributing said 
multimedia content is a real-time network. 

24. The home media server as in claim 23 wherein said real-time network is a 
wireless network. 

25. The home media server as in claim 24 wherein said wireless network is based on 
the IEEE 802.1 lb network protocol. 

26. The home media server as in claim 22 firrther comprising: 
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means for storing said multimedia content for subsequent distribution to said 

multimedia nodes. 

27. The home media server as in claim 22 further comprising: 

means for receiving and transmitting audio over a public switched telephone network 

("PSTN"). 
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